Skip to content

LOO refactoring and adding pred_measure feature#363

Draft
florence-bockting wants to merge 6 commits into
masterfrom
pred_measure
Draft

LOO refactoring and adding pred_measure feature#363
florence-bockting wants to merge 6 commits into
masterfrom
pred_measure

Conversation

@florence-bockting

@florence-bockting florence-bockting commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Fixes #281

This PR is work in progress and the description will be refined over time.

Description

This PR is a larger project with the aim to refactor the loo package and to add new features related to a new pred_measure family.

In the course of refactoring this PR aims to address also the following PRs:

How to navigate this PR

As this is a large, long-running PR it might help to first read through the supporting documents that explain the design, scope, and current status:

  • developer-notes.md: The central reference for this project: current status, open tasks, outstanding design decisions, and known blockers. Start here.
  • pred_measure-formulas.Rmd: An overview of all supported scores and metrics, including their mathematical definitions.
  • pred_measure-family.Rmd: A conceptual introduction to the new pred_measure family and its intended usage.

Acknowledgements

This work builds on an initial prototype of the new pred_measure feature implemented by @VisruthSK under guidance from @jgabry (see VisruthSK/loo-sandbox).

@github-actions

github-actions Bot commented Jun 2, 2026

Copy link
Copy Markdown

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 4f8d324 is merged into master:

  • ✔️loo_function: 1.93s -> 1.93s [-0.55%, +0.44%]
  • ✔️loo_matrix: 1.8s -> 1.8s [-0.49%, +0.67%]
    Further explanation regarding interpretation and methodology can be found in the documentation.

@florence-bockting

florence-bockting commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

Linking here to Issue #179 as it seems to be about loo scores. We should see whether it is still relevant and can be implemented within loo refactoring.

@florence-bockting

florence-bockting commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

There is an open issue requesting a subset() method for psis objects #343 and an (unrelated) open PR #110. Might be worth considering whether we can address this as well in this refactoring PR. If yes, we should

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment